Dynamic accelerator generation and deployment

ABSTRACT

An accelerator deployment tool deploys multiple accelerators to multiple programmable devices, and detects when a new programmable device becomes available. When a first accelerator in a first programmable device is a better match to the new programmable device, the accelerator deployment tool automatically generates an image for the first accelerator for the new programmable device, deploys the image on the new programmable device to generate a second accelerator, changes references to the first accelerator to reference instead the second accelerator, and casts the first accelerator out of the first programmable device.

BACKGROUND 1. Technical Field

This disclosure generally relates to computer systems, and morespecifically relates to hardware accelerators in computer systems.

2. Background Art

The Open Coherent Accelerator Processor Interface (OpenCAPI) is aspecification developed by a consortium of industry leaders. TheOpenCAPI specification defines an interface that allows any processor toattach to coherent user-level accelerators and I/O devices. OpenCAPIprovides a high bandwidth, low latency open interface designspecification built to minimize the complexity of high-performanceaccelerator design. Capable of 25 gigabits (Gbits) per second per lanedata rate, OpenCAPI outperforms the current peripheral componentinterconnect express (PCIe) specification which offers a maximum datatransfer rate of 16 Gbits per second per lane. OpenCAPI provides adata-centric approach, putting the compute power closer to the data andremoving inefficiencies in traditional system architectures to helpeliminate system performance bottlenecks and improve system performance.A significant benefit of OpenCAPI is that virtual addresses for aprocessor can be shared and utilized in an OpenCAPI device, such as anaccelerator, in the same manner as the processor. With the developmentof OpenCAPI, hardware accelerators may now be developed that include anOpenCAPI architected interface.

BRIEF SUMMARY

An accelerator deployment tool deploys multiple accelerators to multipleprogrammable devices, and detects when a new programmable device becomesavailable. When a first accelerator in a first programmable device is abetter match to the new programmable device, the accelerator deploymenttool automatically generates an image for the first accelerator for thenew programmable device, deploys the image on the new programmabledevice to generate a second accelerator, changes references to the firstaccelerator to reference instead the second accelerator, and casts thefirst accelerator out of the first programmable device.

The foregoing and other features and advantages will be apparent fromthe following more particular description, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The disclosure will be described in conjunction with the appendeddrawings, where like designations denote like elements, and:

FIG. 1 is a block diagram of a sample system illustrating how an OpenCoherent Accelerator Processor Interface (OpenCAPI) can be used;

FIG. 2 is a flow diagram of a programmable device with an OpenCAPIinterface that may include one or more hardware accelerators;

FIG. 3 is a block diagram of a computer system that includes a tool fordynamically generating and deploying an accelerator;

FIG. 4 is a flow diagram showing a specific implementation for how theaccelerator image generator in FIG. 3 generates an accelerator imagefrom a code portion;

FIG. 5 is a block diagram of a specific implementation for the codeanalyzer in FIG. 3 that analyzes a computer program and selects a codeportion;

FIG. 6 is a flow diagram of a method for identifying a code portion in acomputer program, dynamically generating and deploying an acceleratorthat corresponds to the code portion, then revising the computer programto replace the code portion with a call to the deployed accelerator;

FIG. 7 is a block diagram showing a first sample computer program withdifferent code portions;

FIG. 8 is a block diagram showing how a code portion can be transformedto HDL, then to an accelerator image, which can be deployed to aprogrammable device to provide an accelerator;

FIG. 9 is a block diagram showing the computer program in FIG. 7 aftercode portion B has been replaced with a call to the accelerator for codeportion B;

FIG. 10 is a block diagram showing a sample accelerator catalog;

FIG. 11 is a flow diagram of a method for deploying an accelerator for acode portion when a catalog of previously-generated accelerators ismaintained;

FIG. 12 is a block diagram showing a second sample computer program withdifferent code portions;

FIG. 13 is a block diagram identifying two code portions in the computerprogram in FIG. 12 that would benefit from an accelerator;

FIG. 14 is a block diagram showing a sample accelerator catalog thatincludes an accelerator that corresponds to code portion Q;

FIG. 15 is a block diagram showing the deployment of an acceleratorimage for code portion Q identified in the catalog in FIG. 14 to aprogrammable device;

FIG. 16 is a block diagram showing the computer program in FIG. 12 aftercode portion Q has been replaced with a call to the accelerator for codeportion Q;

FIG. 17 is a block diagram showing generation of an accelerator imagefrom code portion R in the computer program shown in FIGS. 12 and 16 ;

FIG. 18 is a block diagram showing the deployment of a newly-generatedaccelerator image for code portion R to a programmable device;

FIG. 19 is a is a block diagram showing the computer program in FIG. 16after code portion R has been replaced with a call to the acceleratorfor code portion R;

FIG. 20 is a block diagram of the accelerator catalog 1400 shown in FIG.14 after an entry is created representing the accelerator for codeportion R;

FIG. 21 is a block diagram of a sample computer program;

FIG. 22 is a block diagram of a programmable device that has an OpenCAPIinterface and includes an accelerator for the loop portion in FIG. 21 ,an accelerator for branching tree portion in FIG. 21 , and anaccelerator for lengthy serial portion in FIG. 21 ;

FIG. 23 is a block diagram of the computer program in FIG. 21 after thecode portions have been replaced with calls to correspondingaccelerators;

FIG. 24 is a block diagram of a prior art computer program that callsfunctions in a software library;

FIG. 25 is a flow diagram of a method for replacing calls to thesoftware library with corresponding calls to one or morecurrently-implemented accelerators;

FIG. 26 shows a virtual function table that creates a level ofindirection for calls from the computer program to the software library;

FIG. 27 is a block diagram of the computer program in FIG. 24 after thecalls to the software library have been replaced with calls to thevirtual function table;

FIG. 28 is a block diagram of an accelerator correlation table showingcurrently-implemented accelerators that correspond to functions in thesoftware library;

FIG. 29 is a block diagram of a programmable device showing the threecurrently-implemented accelerators listed in the table in FIG. 28 ;

FIG. 30 shows the virtual function table in FIG. 26 after calls to thesoftware library have been replaced with calls to correspondingaccelerators;

FIG. 31 is a flow diagram of a method for generating a new acceleratorand replacing one or more calls to the software library with one or morecorresponding calls to the new accelerator;

FIG. 32 is a block diagram of a programmable device showing the threepreviously-generated accelerators and the one new accelerator generatedin FIG. 31 ;

FIG. 33 shows the virtual function table in FIGS. 26 and 30 after callsto the software library have been replaced with corresponding calls tothe new accelerator;

FIG. 34 shows a table of possible features in a programmable devicefeature set;

FIG. 35 is a block diagram showing one suitable implementation for theaccelerator image 480 shown in FIG. 4 ;

FIG. 36 is a flow diagram of a method for selecting one or moreprogrammable devices for one or more accelerator images;

FIG. 37 is a table showing how available programmable devices may bespecified;

FIG. 38 is a table showing various match criteria for matching one ormore accelerator images to one or more programmable devices;

FIG. 39 is a block diagram of a sample system for illustrating how oneor more programmable devices may be selected according to theircorresponding feature sets;

FIG. 40 is a block diagram showing example feature sets for theprogrammable devices in FIG. 39 ;

FIG. 41 is a flow diagram of a method for deploying an accelerator to anew programmable device;

FIG. 42 is a block diagram of the system in FIG. 39 after the additionof a new programmable device 1B-2 in Server 1B;

FIG. 43 is a block diagram of the system in FIG. 42 after a new fifthaccelerator has been deployed to the newly added programmable device toreplace the fourth accelerator in programmable device P1-2, which hasbeen cast out of programmable device P1-2;

FIG. 44 is a sample feature set 1B-2 corresponding to the newly-addedprogrammable device 1B-2 shown in FIG. 42 ; and

FIG. 45 is a flow diagram of a method for selecting and deploying one ormore accelerators to the new programmable device.

DETAILED DESCRIPTION

As discussed in the Background Art section above, the Open CoherentAccelerator Processor Interface (OpenCAPI) is a specification thatdefines an interface that allows any processor to attach to coherentuser-level accelerators and I/O devices. Referring to FIG. 1 , a samplecomputer system 100 is shown to illustrate some of the concepts relatedto the OpenCAPI interface 150. A processor 110 is coupled to a standardmemory 140 or memory hierarchy, as is known in the art. The processor iscoupled via a PCIe interface 120 to one or more PCIe devices 130. Theprocessor 110 is also coupled via an OpenCAPI interface 150 to one ormore coherent devices, such as accelerator 160, coherent networkcontroller 170, advanced memory 180, and coherent storage controller 190that controls data stored in storage 195. While the OpenCAPI interface150 is shown as a separate entity in FIG. 1 for purposes ofillustration, instead of being a separate interface as shown in FIG. 1 ,the OpenCAPI interface 150 can be implemented within each of thecoherent devices. Thus, accelerator 160 may have its own OpenCAPIinterface, as may the other coherent devices 170, 180 and 190. One ofthe significant benefits of OpenCAPI is that virtual addresses for theprocessor 110 can be shared with coherent devices that are coupled to orinclude an OpenCAPI interface, permitting them to use the virtualaddresses in the same manner as the processor 110.

Referring to FIG. 2 , a programmable device 200 represents any suitableprogrammable device. For example, the programmable device 200 could bean FPGA or an ASIC. An OpenCAPI interface 210 can be implemented withinthe programmable device. In addition, one or more accelerators can beimplemented in the programmable device 200. FIG. 1 shows by way ofexample accelerator 1 220A, accelerator 2 220B, . . . , accelerator N220N. In the prior art, a human designer would determine what type ofaccelerator is needed based on a function that needs to be acceleratedby being implemented in hardware. The accelerator function could berepresented, for example, in a hardware description language (HDL).Using known tools, the human designer can then generate an acceleratorimage that corresponds to the HDL. The accelerator image, once loadedinto the programmable device such as 200 in FIG. 2 , creates anaccelerator in the programmable device that may be called as needed byone or more computer programs to provide the hardware accelerator(s).

An accelerator deployment tool deploys multiple accelerators to multipleprogrammable devices, and detects when a new programmable device becomesavailable. When a first accelerator in a first programmable device is abetter match to the new programmable device, the accelerator deploymenttool automatically generates an image for the first accelerator for thenew programmable device, deploys the image on the new programmabledevice to generate a second accelerator, changes references to the firstaccelerator to reference instead the second accelerator, and casts thefirst accelerator out of the first programmable device.

Referring to FIG. 3 , a computer system 300 is one suitableimplementation of a computer system that includes an acceleratordeployment tool that automatically selects one or more a programmabledevices based on feature sets compared to resource requirements, andautomatically deploys one or more accelerator images on the programmabledevice(s), as described in more detail below. Server computer system 300is an IBM POWER9 computer system. However, those skilled in the art willappreciate that the disclosure herein applies equally to any computersystem, regardless of whether the computer system is a complicatedmulti-user computing apparatus, a single user workstation, a laptopcomputer system, a tablet computer, a phone, or an embedded controlsystem. As shown in FIG. 3 , computer system 300 comprises one or moreprocessors 310, one or more programmable devices 312, a main memory 320,a mass storage interface 330, a display interface 340, and a networkinterface 350. These system components are interconnected through theuse of a system bus 360. Mass storage interface 330 is used to connectmass storage devices, such as local mass storage device 355, to computersystem 300. One specific type of local mass storage device 355 is areadable and writable CD-RW drive, which may store data to and read datafrom a CD-RW 395. Another suitable type of local mass storage device 355is a card reader that receives a removable memory card, such as an SDcard, and performs reads and writes to the removable memory. Yet anothersuitable type of local mass storage device 355 is universal serial bus(USB) that reads a storage device such a thumb drive.

Main memory 320 preferably contains data 321, an operating system 322, acomputer program 323, an accelerator deployment tool 324, and anaccelerator catalog 329. Data 321 represents any data that serves asinput to or output from any program in computer system 300. Operatingsystem 322 is a multitasking operating system, such as AIX or LINUX.Computer program 323 represents any suitable computer program, includingwithout limitations an application program, an operating system,firmware, a device driver, etc. The accelerator deployment tool 324preferably includes a code analyzer 325, an accelerator image generator327, and an accelerator implementer 328. The code analyzer 325 analyzesthe computer program 324 as it runs to determine its run-timeperformance. One suitable way for code analyzer 325 to analyze thecomputer program is using known techniques for monitoring the run-timeperformance of a computer program. For example, tools exist in the artthat allow real-time monitoring of the run-time performance of acomputer program using a monitor external to the computer program thatdetects, for example, which addresses are being executed by theprocessor 310 during the execution of the computer program 323. Othertools known as profilers allow inserting instrumentation code into acomputer program, which is code that increments different counters whendifferent branches of the computer program are executed. The values ofthe counters can be analyzed to determine the frequency of executingeach portion of the computer program. The code analyzer 325, afteranalyzing the run-time performance of the computer program, identifies acode portion 326, which is a portion of code in the computer program323, that will be improved from being deployed to a hardware acceleratorto enhance the run-time performance of the computer program 323.

The accelerator image generator 327 dynamically generates an acceleratorimage corresponding to the code portion 326 in the computer program 323identified by the code analyzer 325. The accelerator image generator 327may generate an accelerator image from code portion 326 using anysuitable method. For example, the accelerator image generator 327 couldgenerate an equivalent hardware description language (HDL)representation of the code portion 326, then synthesize the HDLrepresentation into a suitable accelerator image for the programmabledevice 312. The accelerator implementer 328 preferably takes anaccelerator image generated by the accelerator image generator 327, anduses the accelerator image to program the programmable device 312,thereby generating a hardware accelerator 314 in programmable device 312that corresponds to the code portion 326.

In a first implementation, the accelerator deployment tool 324dynamically generates an accelerator image corresponding to the codeportion 326 of the computer program 323, then programs the programmabledevice with the accelerator image so the programmable device includes ahardware accelerator that corresponds to the code portion 326. In asecond implementation, an accelerator catalog 329 is provided andmaintained. The accelerator catalog 329 preferably includes a listing ofpreviously-generated accelerators. In the second implementation, theaccelerator deployment tool 324 first checks the accelerator catalog 329to see if a previously-generated accelerator is available for the codeportion 326. If so, the accelerator deployment tool 324 deploys apreviously generated accelerator image identified in the acceleratorcatalog. If not, the accelerator deployment tool 324 dynamicallygenerates an accelerator image as described above, then loads the imageinto the programmable device 312 to provide the accelerator 314 thatcorresponds to the code portion 326.

Computer system 300 utilizes well known virtual addressing mechanismsthat allow the programs of computer system 300 to behave as if they onlyhave access to a large, contiguous address space instead of access tomultiple, smaller storage entities such as main memory 320 and localmass storage device 355. Therefore, while data 321, operating system322, computer program 323, accelerator deployment tool 324, andaccelerator catalog 329 are shown to reside in main memory 320, thoseskilled in the art will recognize that these items are not necessarilyall completely contained in main memory 320 at the same time. It shouldalso be noted that the term “memory” is used herein generically to referto the entire virtual memory of computer system 300, and may include thevirtual memory of other computer systems coupled to computer system 300.

Processor 310 may be constructed from one or more microprocessors and/orintegrated circuits. Processor 310 could be, for example, one or morePOWER9 microprocessors. Processor 310 executes program instructionsstored in main memory 320. Main memory 320 stores programs and data thatprocessor 310 may access. When computer system 300 starts up, processor310 initially executes the program instructions that make up operatingsystem 322. Processor 310 also executes the computer program 323 and theaccelerator deployment tool 324.

Programmable device(s) 312 can be any suitable programmable logic devicethat can be dynamically programmed by the processor 310. Examples ofknown suitable programmable logic devices include field-programmablegate arrays (FPGAs). However, the programmable device 312 broadlyincludes any programmable logic device that allows the processor 310 todynamically program the programmable device 312, including knowntechnologies as well as technologies that are developed in the future.

Although computer system 300 is shown to contain only a single processorand a single system bus, those skilled in the art will appreciate thatan accelerator deployment tool as described herein may be practicedusing a computer system that has multiple processors and/or multiplebuses. In addition, the interfaces that are used preferably each includeseparate, fully programmed microprocessors that are used to off-loadcompute-intensive processing from processor 310. However, those skilledin the art will appreciate that these functions may be performed usingI/O adapters as well.

Display interface 340 is used to directly connect one or more displays365 to computer system 300. These displays 365, which may benon-intelligent (i.e., dumb) terminals or fully programmableworkstations, are used to provide system administrators and users theability to communicate with computer system 300. Note, however, thatwhile display interface 340 is provided to support communication withone or more displays 365, computer system 300 does not necessarilyrequire a display 365, because all needed interaction with users andother processes may occur via network interface 350.

Network interface 350 is used to connect computer system 300 to othercomputer systems or workstations 375 via network 370. Computer systems375 represent computer systems that are connected to the computer system300 via the network interface 350. Network interface 350 broadlyrepresents any suitable way to interconnect electronic devices,regardless of whether the network 370 comprises present-day analogand/or digital techniques or via some networking mechanism of thefuture. Network interface 350 preferably includes a combination ofhardware and software that allows communicating on the network 370.Software in the network interface 350 preferably includes acommunication manager that manages communication with other computersystems 375 via network 370 using a suitable network protocol. Manydifferent network protocols can be used to implement a network. Theseprotocols are specialized computer programs that allow computers tocommunicate across a network. TCP/IP (Transmission ControlProtocol/Internet Protocol) is an example of a suitable network protocolthat may be used by the communication manager within the networkinterface 350. In one suitable implementation, the network interface 350is a physical Ethernet adapter.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

FIG. 4 illustrates details of one suitable implementation of theaccelerator image generator 327 shown in FIG. 3 . The accelerator imagegenerator 327 takes as input the code portion 326 shown in FIGS. 3 and 4. A code to HDL generator 410 preferably converts the code portion 326to a corresponding representation of the code portion in a hardwaredescription language (HDL), shown in FIG. 4 as HDL for code portion 420.Known suitable hardware description languages include VHDL or Verilog,but any suitable hardware description language could be used. There areknown software tools for generating an HDL representation of computercode. For example, Xilinx's Vivado High Level Synthesis is a softwaretool that converts code written in the C programming language to HDL.This type of tool is often referred to in the art as a “C to HDL” toolor a “C to RTL” tool, where RTL refers to the Register Transfer Levelrepresentation of a code portion needed to implement the code portion inhardware. The Code to HDL Generator 410 in FIG. 4 could be a knownsoftware tool, or could be a software tool specifically designed for theaccelerator image generator 327.

The HDL for the code portion 420 is fed into one or more processes thatmay include both synthesis and simulation. The synthesis process 430 isshown in the middle portion of FIG. 4 in steps 432, 434, 436, 438 and440. The simulation process 450 is shown in the lower portion of FIG. 4in steps 452, 454 and 460. The HDL for code portion 420 may be fed intothe synthesis block 432, which determines which hardware elements areneeded. The place and route block 434 determines where on theprogrammable device to put the hardware elements, and how to routeinterconnections between those hardware elements. Timing analysis 436analyzes the performance of the accelerator after the hardware elementshave been placed and interconnections have been routed in block 434.Test block 438 runs tests on the resulting accelerator image todetermine whether timing and performance parameters are satisfied. Thetest block 438 feeds back to debug block 440 when the design of theaccelerator still needs improvement. This process may iterate severaltimes.

The simulation process 450 takes in the HDL for the code portion 420,and performs a computer simulation to determine its functionality. Asimulated test block 454 determines whether the simulated designfunctions as needed. The simulated test block 454 feeds back to a debugblock 460 when the design of the accelerator still needs improvement.

The accelerator image generator 327 may include either the synthesisblock 430, the simulation block 450, or both. In the most preferredimplementation, the accelerator image generator 327 includes both thesynthesis block 430 and the simulation block 450. The synthesis processcan be very time-consuming. The simulation block is typically muchfaster in testing the design of the HDL than the synthesis block. Whenboth synthesis 430 and simulation 450 are both present, the acceleratorimage generator can use both of these in any suitable way orcombination. For example, the simulation block 450 could be usedinitially to iterate a few times on the design, and when the design ismostly complete, the mostly-completed design could be fed into thesynthesis block 430. In another implementation, the synthesis andsimulation blocks could function in parallel and cooperate until thegeneration of the accelerator image is complete. Regardless of thespecific process used, the accelerator image generator 327 generates forthe code portion 326 an accelerator image 480 that corresponds to thecode portion 326. Once the accelerator image 480 has been generated, theaccelerator implementer 328 in FIG. 3 can load the accelerator image 480into a programmable device 312 to produce an accelerator 314corresponding to the code portion 326. The accelerator 314 in theprogrammable device 312 may then be called by the computer program inplace of the code portion 326.

Some details of one possible implementation for the code analyzer 325 inFIG. 3 are shown in FIG. 5 . The code analyzer 325 can include a codeprofiler 510 that is used to profile the computer program. Profiling isdone by the code profiler 510 preferably inserting instrumentation codeinto the computer program to generate profile data 520 as the computerprogram runs. The profile data 520 indicates many possible features ofthe computer program, including the frequency of executing differentportions, the number or loop iterations, exceptions generated, datademand, bandwidth, time spent in a critical portion, etc. Softwareprofilers are very well-known in the art, and are therefore notdiscussed in more detail here. For our purposes herein, suffice it tosay the code profiler 510 generates profile data 520 that indicatesrun-time performance of the computer program being profiled.

The code analyzer 325 additionally includes a code selection tool 530that identifies a code portion 326 that will be improved from beingimplemented in a hardware accelerator. Any suitable code portion couldbe identified according to any suitable criteria, algorithm orheuristic. For example, a portion of the code that performsfloating-point calculations could be identified so that a correspondingfloating-point accelerator could be generated to perform thefloating-point calculations in the code. A portion of the code thatperforms a search of a database could be identified so a correspondingdatabase search accelerator could be generated to replace the databasesearch. A portion of the code that performs a specific function, such asdata compression, XML parsing, packet snooping, financial riskcalculations, etc., could also be identified. Of course, other codeportions could be identified within the scope of the disclosure andclaims herein. The code selection tool 530 can use any suitablecriteria, algorithm or heuristic, whether currently known or developedin the future, to identify code portion 326. Once the code portion 326in the computer program has been identified, a corresponding acceleratormay be dynamically generated.

Referring to FIG. 6 , a method 600 starts by running the computerprogram (step 610). The run-time performance of the computer program isanalyzed (step 620). This can be done, for example, by the code analyzer325 shown in FIGS. 3 and 5 and discussed above. A code portion in thecomputer program is identified to implement in an accelerator (step630). An accelerator image for the code portion is generated (step 640).The accelerator image is deployed to a programmable device (step 650).The computer program is then revised to replace the code portion with acall to the deployed accelerator (step 660). At this point, the deployedaccelerator will perform the functions in hardware that were previouslyperformed by the code portion, thereby improving the run-timeperformance of the computer program. Note that method 600 loops back tostep 610 and continues, which means method 600 can iterate tocontinuously monitor the computer program and deploy accelerators, asneeded, to improve performance of the computer program.

Some examples are now provided to illustrate the concepts discussedabove. FIG. 7 shows a sample computer program 700 that includes multiplecode portions, shown in FIG. 7 as code portion A 710, code portion B720, code portion C 730, . . . , code portion N 790. We assume codeportion B 720 is identified as a code portion that will be improved frombeing implemented in a hardware accelerator. Code portion B 720 is thenconverted to a corresponding HDL representation 810, as shown in FIG. 8. The HDL for code portion B 810 is then used to generate an acceleratorimage for code portion B 820. This could be done, for example, using themethod shown in FIG. 4 , or using any other suitable method. Once theaccelerator image for code portion B 820 has been generated, theaccelerator image is loaded into a programmable device 830 to generatethe accelerator for code portion B 850. Programmable device 830 is onesuitable implementation for the programmable device 312 shown in FIG. 3, and preferably includes an OpenCAPI interface 840.

Once the accelerator is deployed in the programmable device 830, thecode portion B in the computer program is deleted and replaced by a callto the accelerator for code portion B 910 shown in FIG. 9 . In the mostpreferred implementation, the accelerator for code portion B includes areturn to the code that called it once the processing in the acceleratorfor code portion B is complete. In this manner the computer program 900,when it needs to execute what was previously code portion B, will make acall to the accelerator for code portion B, which will perform theneeded functions in hardware, then return to the computer program. Inthis manner a suitable accelerator may be automatically generated for anidentified code portion to increase the run-time performance of thecomputer program.

In a first implementation, an accelerator may be dynamically generatedto improve the performance of a computer program, as shown in FIGS. 4-9and described above. In a second implementation, once an accelerator isdynamically generated, it can be stored in a catalog so it may be reusedwhen needed. FIG. 10 shows a sample accelerator catalog 1000, which isone suitable implementation for the accelerator catalog 329 shown inFIG. 3 . An accelerator catalog may include any suitable data orinformation that may be needed for an accelerator or the correspondingcode portion. For the specific example shown in FIG. 10 , acceleratorcatalog includes each of the following fields: Name, Location, LeastRecently Used (LRU), Most Recently Used (MRU), Dependencies,Capabilities, Latency, and Other Characteristics. The Name fieldpreferably includes a name for the accelerator. The name field may alsoinclude a name for a code portion that corresponds to the accelerator.The location field preferably specifies a path that identifies thelocation for the accelerator image. While the accelerator image could bestored in the catalog 1000, in the most preferred implementation thecatalog 1000 instead includes a path to storage external to theaccelerator catalog 1000 where the accelerator image is stored. Theleast recently used (LRU) field could include the time when theaccelerator was used the first time. In the alternative, the LRU fieldcould include a flag that is set when the accelerator is the leastrecently used of all the accelerators in the catalog. The most recentlyused (MRU) field could include the time when the accelerator was lastused. In the alternative, the MRU field could include a flag that is setwhen the accelerator is the most recently used of all the acceleratorsin the catalog. The error rate field provides a suitable error rate forthe accelerator, and can be expressed in any suitable way. For theexample in FIG. 10 , the error rate is expressed as a number X of errorsper 100 runs of the accelerator. The error rate field could include anysuitable error information that could be, for example, dynamicallymonitored so an increase in the error rate could result in anotification to take corrective action. The dependencies field mayindicate any dependencies the accelerator may have. For example, thedependencies field could specify the specific programmable device theaccelerator was designed for. The dependencies field could also specifyany dependencies on other accelerators. Thus, accelerator Acc1 in FIG.10 has a dependency on Acc2, which means Acc1 needs Acc2 to also beimplemented. The capabilities field can provide any suitable indicationof the capabilities of the accelerator. In the two entries shown in FIG.10 , the capabilities are shown as floating-point (FP) Unit for Acc1 andGraphics for AccN. Note, however, the capabilities can be indicated inany suitable way. For example, the capabilities could include aspecification of the code portion for which the accelerator wasimplemented. A separate index could be maintained that correlates eachcode portion to its corresponding accelerator, along with a descriptoror other data that describes attributes of the code portion. Thecapabilities field could include any suitable information, such as apointer to the index, so the code portion corresponding to theaccelerator could be easily identified.

The latency field preferably specifies average latency for theaccelerator. For the example shown in FIG. 10 , Acc1 has a latency of1.0 microseconds while accelerator AccN has a latency of 500nanoseconds. Latency could represent, for example, the time required forthe accelerator to perform its intended function. The othercharacteristics field can include any other suitable information or datathat describes or otherwise identifies the accelerator, itscharacteristics and attributes, and the code portion corresponding tothe accelerator. For the two sample entries in FIG. 10 , the othercharacteristics field indicates Acc1 includes a network connection, andAccN has an affinity to Acc5, which means AccN should be placed in closeproximity to Acc5 on the programmable device, if possible. The variousfields in FIG. 10 are shown by way of example, and it is within thescope of the disclosure and claims herein to provide an acceleratorcatalog with any suitable information or data.

Referring to FIG. 11 , a method 1100 in accordance with the secondimplementation begins by running the computer program (step 1110). Therun-time performance of the computer program is analyzed (step 1120).One or more code portions in the computer program that will be improvedby use of a hardware accelerator are identified (step 1130). One of theidentified code portions is selected (step 1140). When there is apreviously-generated accelerator in the accelerator catalog for theselected code portion (step 1150=YES), the previously-generatedaccelerator image is deployed to the programmable device (step 1160) toprovide the accelerator. The computer program is then revised to replacethe selected code portion with a call to the accelerator (step 1162).When there is no previously-generated accelerator in the catalog for theselected code portion (step 1150=NO), an accelerator image for theselected code portion is dynamically generated (step 1170), theaccelerator image is deployed to a programmable device (step 1172) thecomputer program is revised to replace the code portion with a call tothe newly deployed accelerator (step 1174), and the accelerator isstored to the accelerator catalog (step 1176). When the acceleratorimage is stored within the catalog entry, step 1176 write theaccelerator image to the catalog. When the accelerator image is storedin storage external to the catalog, step 1176 stores the acceleratorimage to the external storage and writes an entry to the acceleratorcatalog that includes a path to the accelerator image in the externalstorage.

When there are more identified code portions (step 1180=YES), method1100 loops back to step 1140 and continues. When there are no moreidentified code portions (step 1180=NO), method 1100 loops back to step1120 and continues. This means method 1100 most preferably continuouslymonitors the computer program and dynamically generates and/or deploysaccelerators as needed to improve the run-time performance of thecomputer program.

An example is now provided to illustrate the concepts in FIG. 11 thatrelate to the second preferred implementation. FIG. 12 shows a samplecomputer program 1200 that includes many code portions, represented inFIG. 12 as code portion P 1210, code portion Q 1220, code portion R1230, . . . , code portion Z 1290. We assume steps 1110, 1120 and 1130in FIG. 11 are performed. In step 1130, we assume code portion Q 1220and code portion R 1230 are identified as code portions that will beimproved by implementing these code portions in an accelerator, as shownin table 1300 in FIG. 13 . We further assume we have an acceleratorcatalog 1400 that is one suitable implementation for the acceleratorcatalog 329 shown in FIG. 3 . Accelerator catalog 1400 has a singleentry for AccQ, which we assume is an accelerator for code portion Q1220 that was generated previously. Because the accelerator for codeportion Q was previously-generated, the corresponding accelerator imagecan be used without having to generate the accelerator image anew. Weassume code portion Q 1220 is selected in step 1140. There is apreviously-generated accelerator in the catalog for code portion Q (step1150=YES), so the previously-generated accelerator image correspondingto code portion Q 1510 is deployed to the programmable device (step1160), as shown in FIG. 15 . Deploying the accelerator image for codeportion Q 1510 identified in the catalog to the programmable device 1520results in implementing the accelerator for code portion Q 1540 in theprogrammable device 1520. The accelerator for code portion Q 1540 maythen be called by the computer program to perform the functions ofprevious code portion Q in hardware, thereby increasing the run-timeperformance of the computer program. The programmable device 1520 is onesuitable example of a programmable device 312 shown in FIG. 3 , andpreferably includes an OpenCAPI interface 1530.

The computer program is then revised to replace the selected codeportion Q 1220 with a call to the accelerator for code portion Q (step1162). FIG. 16 shows the computer program 1200 in FIG. 12 after the codeportion Q has been replaced with the call to the accelerator for codeportion Q, as shown at 1610 in FIG. 16 . Thus, computer program 1600,instead of executing code portion Q, instead invokes the accelerator forcode portion Q 1540 in the programmable device 1520 to increase therun-time performance of the computer program.

There is still an identified code portion (step 1180=YES), namely codeportion R shown in FIG. 13 , so method 11 in FIG. 11 loops back to step1140, where code portion R 1230 is selected (step 1140). There is nopreviously-generated accelerator in the catalog 1400 shown in FIG. 14for code portion R (step 1150=NO), so an accelerator image isdynamically generated for code portion R (step 1170). This isrepresented in FIG. 17 , where the code portion R 1230 is used togenerate HDL for code portion R 1710, which is used to generate theaccelerator image for code portion R 1720. The accelerator image forcode portion R 1720, which was newly dynamically generated, is thendeployed to the programmable device (step 1172). This is shown in FIG.18 , where the programmable device 1520 that already includesaccelerator for code portion Q 1540 is loaded with the accelerator imagefor code portion R 1720 to generate the accelerator for code portion R1810. The computer program is then revised to replace code portion Rwith the call to the accelerator for code portion R (step 1174), asshown at 1910 in FIG. 19 . The accelerator for code portion R is alsostored in the accelerator catalog (step 1176), resulting in theaccelerator catalog 1400 containing entries AccQ and AccR correspondingto two accelerators, as shown in FIG. 20 .

A more specific example is shown in FIGS. 21 and 22 . For this examplewe assume a computer program called Sample1 2100 includes threedifferent code portions of interest, namely a loop portion 2110, abranching tree portion 2120, and a lengthy serial portion 2130. Loopportion 2110 is representative of a code portion that is a loop that canbe unrolled because each iteration is largely independent from otheriterations. Due to the independence of each iteration, the loop can beunrolled, and the loop function can be deployed to an accelerator soeach iteration will run in parallel in hardware. Financial riskcalculations sometimes include code portions such as loop portion 2110.Running different iterations of the loop in parallel in a hardwareaccelerator increases the run-time performance of the Sample1 computerprogram.

Computer program Sample1 2100 also includes a branching tree portion2120. We assume for this example branching tree portion 2120 operates onone or more relatively deep branching trees. In this case, the branchingtree portion 2120 can be deployed to an accelerator so each branch ofthe branching tree will run in parallel in hardware, the branchselection criteria will be calculated, and at the final stage of thelogic, the result will be selected from the selected branch. Runningdifferent branches of the branching tree in parallel in a hardwareaccelerator increases the run-time performance of the Sample1 computerprogram.

Computer program Sample1 2100 also includes a lengthy serial portion2130. We assume for this example the lengthy serial portion 2130 can beshortened by leveraging unique hardware capabilities in an accelerator.Some math functions, for example, could by lengthy serial portions thatcould be implemented in an accelerator. Running a lengthy serial portionin hardware increases the run-time performance of the Sample1 computerprogram.

We assume the code portions in FIG. 21 are identified according toprofile data 520 generated by the code profiler 510 in FIG. 5 . Thecriteria used by the code selection tool 530 to select the code portions2110, 2120 and 2130, which are examples of code portion 326 in FIGS. 3and 5 , may be any suitable criteria. The three example code portions2110, 2120 and 2130 in FIG. 21 as described above indicate suitablecriteria that could be used by the code selection tool 530 to selectcode portions 2110, 2120 and 2130 to be implemented in one or moreaccelerators. Of course, the claims and disclosure herein expresslyextend to any suitable criteria for the code selection tool 530 toselect one or more code portions 326 to be implemented in one or moreaccelerators.

FIG. 22 shows a programmable device 2220 that has an OpenCAPI interface2230 and includes an accelerator for loop portion 2240, an acceleratorfor branching tree portion 2250, and an accelerator for lengthy serialportion 2260. While these three accelerators are shown to be implementedin the same programmable device 2220 in FIG. 22 , one skilled in the artwill recognize these could be implemented in separate programmabledevices as well.

FIG. 23 shows the computer program Sample1 2100 after the code portionsshown in FIG. 21 are replaced with calls to the hardware acceleratorsshown in FIG. 22 . Thus, loop portion 2110 in FIG. 21 has been replacedby a call to the accelerator for loop portion 2310; the branching treeportion 2320 in FIG. 21 has been replaced by a call to the acceleratorfor the branching tree portion 2320; and the lengthy serial portion 2130in FIG. 21 has been replaced by a call to the accelerator for thelengthy serial portion 2330. Because the Sample1 computer program 2100in FIG. 23 now includes calls to hardware accelerators, the run-timeperformance of the computer program 2100 is increased.

FIG. 24 shows a prior art computer program 2400 that includes calls tofunctions in a software library 2410. Software libraries are verywell-known in the art, and provide common functions that programmers canuse instead of having to code these common functions. For example,functions that perform compression, graphics operations and XML, parsingcould be included in a software library. The computer program 2400includes code portion D 2420, code portion E 2422, code portion F 2424,possibly other code portions not shown, through code portion L 2428.Software library 2410 includes functions L1 2430, L2 2432, L3 2434, L42436, possibly other functions, through LN 2450. Code portion D 2420 incomputer program 2400 includes a call to function L1 2430 in softwarelibrary 2410. Code portion F 2424 includes a call to function L4 2436 insoftware library 2410. Code portion L 2428 includes a call to functionL2 2432 in software library 2410.

Referring to FIG. 25 , a method 2500 is preferably performed by theaccelerator deployment tool 324 in FIG. 3 . Calls in the computerprogram to the software library are determined (step 2510). A virtualfunction table is built that includes the calls to the software library(step 2520). The available accelerators that are currently implementedin one or more programmable devices are determined (step 2530). Calls inthe software library that correspond to a currently-implementedaccelerator are determined (step 2540). One or more function calls tothe software library in the virtual function table are then replacedwith one or more corresponding calls to a correspondingcurrently-implemented accelerator (step 2550). Note that method 2500then loops back to step 2510, indicating this method can continuouslyperforms its functions as accelerators are deployed or removed.

One specific implementation of a virtual function table is shown at 2600in FIG. 26 . The virtual function table 2600 lists calls from thecomputer program that were previously made directly to the softwarelibrary, and creates a level of indirection so those calls can be madeto an accelerator instead when possible. The calls in the computerprogram 2400 in FIG. 24 have been replaced by calls to the functions inthe virtual function table 2600, as shown in computer program 2700 inFIG. 27 . Thus, the call to L1 is replaced with a call to F1; the callto L4 is replaced with a call to F4; and the call to L2 is replaced witha call to F2. The virtual function table 2600 indicates which functionsto call for each call from the computer program. When the virtualfunction table is initially built, each call from the computer programis mapped to the corresponding call to the software library. Themodified computer program 2700 and virtual function table 2600 thusprovide similar functionality as shown in FIG. 24 , but with a level ofindirection. Thus, code portion D 2720 calls function F1 in the virtualfunction table 2600, which generates a call to L1 in the softwarelibrary. Code portion F 2724 calls function F4 in the virtual functiontable 2600, which generates a call to L4 in the software library. Codeportion L 2728 calls function F2 in the virtual function table, whichgenerates a call to L2 is the software library. We see from this simpleexample that when the virtual function table is initially built, itprovides similar function as shown in FIG. 24 , namely, each call to thevirtual function table results in a corresponding call to the softwarelibrary.

FIG. 28 shows an accelerator correlation table 2800. We assume for thisexample that three accelerators have been deployed, namely Acc1, Acc2and Acc3. We assume these accelerators correspond to three functions inthe software library. Thus, Acc1 corresponds to library function L4;Acc2 corresponds to library function L1; and Acc3 corresponds to libraryfunction L2, as indicated in FIG. 28 . The correlation between theaccelerators and library functions can be determined in any suitableway, including a user manually generating entries to the acceleratorcorrelation table, or the accelerator deployment tool automaticallydetermining the correlation between accelerators and library functions.For accelerators manually generated by a user, the user could use thesame library name and function names, thus allowing a code linker toautomatically detect the accelerator and create the call to theaccelerator instead of to the software library. Similarly,automatically-generated accelerators could use the same library name andfunction names, allowing the code linker to function in similar fashionto automatically detect the accelerator and create the call to theaccelerator instead of to the software library. In a differentimplementation the accelerator could include data that characterizes itsfunctions, thereby allowing the accelerator to be queried to determinethe functions it supports, which information could be used to replacecalls to the software library with calls to the accelerator instead.

FIG. 29 shows a programmable device 2900 that includes an OpenCAPIinterface 2230 and the three accelerators Acc1, Acc2 and Acc3 referencedin FIG. 28 . These three accelerators 2910, 2920 and 2930 arecurrently-implemented accelerators because they already exist in theprogrammable device 2900. FIG. 29 also shows available resources 2950 onthe programmable device 2900 that have not yet been used.

We now consider method 2500 in FIG. 25 with respect to the specificexample in FIGS. 26-29 . Steps 2510 and 2520 build the virtual functiontable 2600 in FIG. 26 . Step 2530 determines Acc1 2910, Acc2 2920 andAcc3 2930 are currently implemented in a programmable device 2900 andare available for use. Step 2540 reads the accelerator correlation table2800 to determine that Acc1 corresponds to library function L4; Acc2corresponds to library function L1; and Acc3 corresponds to libraryfunction L2. As discussed above, these library functions could befunctions that perform compression, graphics operations, XML parsing, orany other suitable library functions. Step 2550 then replaces calls tothe software library in the virtual function table with calls to thecurrently-implemented accelerators, as shown in the virtual functiontable 2600 in FIG. 30. The virtual function table thus provides a levelof indirection that allows dynamically replacing a call to the softwarelibrary with a call to an accelerator without the computer program beingaware the software library function has been implemented in anaccelerator. The result is improved run-time performance of the computerprogram in a way that is transparent to the computer program.

In an alternative embodiment, not only can currently-implementedaccelerators be used to replace calls to software library functions, buta new accelerator can be dynamically generated to replace a call to asoftware library function as well. Referring to FIG. 31 , when a call tothe software library cannot be implemented in a new accelerator (step3110=NO), method 3100 loops back to step 3110 and continues until a callto the software library could be implemented in a new accelerator (step3110=YES). One factor that comes into play in deciding whether a call tothe software library could be implemented in a new accelerator is theavailable resources on one or more programmable devices. For example, ifthe available resources 2950 in FIG. 29 provide sufficient resources forimplementing a call to the software library in a new accelerator thatcould be deployed to the available resources 2950, step 3110 could beYES. An accelerator image for the new accelerator is dynamicallygenerated (step 3120). One suitable way to dynamically generate a newaccelerator image is using the process in FIG. 4 discussed in detailabove. Of course, other ways to dynamically generate an acceleratorimage are also within the scope of the disclosure and claims herein. Theaccelerator image dynamically generated in step 3120 is then deployed toa programmable device to create the new accelerator (step 3130). One ormore calls to the software library in the virtual function table arereplaced with corresponding one or more calls to the new accelerator(step 3140). Method 3100 then loops back to step 3110 and continues,indicating method 3100 can continuously monitor and function to createnew accelerators, as needed.

We continue with the same example in FIGS. 26-30 in discussing method3100 in FIG. 31 . We assume for this specific example that step 3110determines the call to L3 in the software library could be implementedin a new accelerator (step 3110=YES). We assume an accelerator image forthe new accelerator called Acc4 is generated in step 3120, then deployedto a programmable device in step 3130. We assume the image for Acc4 isdeployed to the same programmable device 2900 shown in FIG. 29 ,resulting in the programmable device 2900 including Acc1 2910, Acc22920, Acc3 2930, and Acc4 3240, as shown in FIG. 32 . Note the availableresources 3250 are less than in FIG. 29 because Acc4 has used some ofthose resources. Step 3140 in FIG. 31 then replaces the call to L4 inthe virtual function table with a call to Acc4, as shown in FIG. 33 . Atthis point, when the computer program calls function F4 in the virtualfunction table 2600, Acc4 will be called to perform this functioninstead of performing the function via a call to the software library.

The accelerator deployment tool can select a programmable device fordeploying one or more accelerator images based on the resourcerequirements of the accelerator image(s) compared to feature sets foravailable programmable devices. Referring to FIG. 34 , a table 3400shows examples of features that could be included to characterize aprogrammable device. A collection of features that characterizes aprogrammable device is referred to herein as a “feature set” for theprogrammable device. As shown in FIG. 34 , a feature set for aprogrammable device may include one or more digital features, one ormore analog features, and one or more other features. The digitalfeatures may include one or more of the following: logic blocks 3402;flip-flops 3404; memory 3406; input/output (I/O) blocks 3408; processors3410; Digital Signal Processing (DSP) slices 3412; network interfaces3414; multipliers 3416; high-speed I/O logic 3418; transceivers 3420;Ethernet Media Access Control (MAC) 3422; bus controllers 3424; externalmemory controllers 3426; disk controllers 3428; and other digitalfeatures 3430. Note the digital features shown in FIG. 34 are shown byway of example, and are not limiting. Other digital features 3430 mayinclude any digital features not listed in FIG. 34 , whether currentlyknown or developed in the future.

Note that some programmable devices provide “hard blocks”, which aretypically blocks of logic on the device that are fixed, i.e., notprogrammable, but that may be used by connecting these blocks toprogrammable blocks on the device. Any of the digital features shown inFIG. 34 could be programmable logic blocks or could be hard blocks.Often, hard blocks are higher-level functions, such as processors 3410,DSP slices 3412, high-speed I/O logic 3418, transceivers 3420, EthernetMAC 3422, bus controller 3424, and external memory controller 3426. Ahard block is also referred to herein as a fixed feature of aprogrammable device.

As shown in FIG. 34 , the analog features in a feature set for aprogrammable device may include programmable slew rate on output pins3440; oscillators 3442; phased-locked loops (PLLs) 3444; differentialcomparators 3446; analog-to-digital converters (ADCs) 3448;digital-to-analog converters (DACs) 3450; analog signal conditioningblocks 3452; and other analog features 3454. Note the analog featuresshown in FIG. 34 are shown by way of example, and are not limiting.Other analog features 3454 may include any analog features not listed inFIG. 34 , whether currently known or developed in the future.

The feature set for a programmable device may include other features, asshown in FIG. 34 . Other features may include, for example, estimatedpower rating 3460; estimated speed rating 3462; technology type 3464;manufacturer 3466; family 3468; and other criteria 3470. Estimated powerrating 3460 can characterize the estimated power consumption of theprogrammable device in any suitable way. For example, power ratingcategories of 1 through 5 could be defined, with each programmabledevice being assigned a power rating number based on its estimated powerconsumption. Of course, the estimated power rating 3460 could be definedin other ways as well. The estimated speed rating 3462 can characterizethe estimated speed of the programmable device in any suitable way. Forexample, speed rating categories of 1 through 4 could be defined, witheach programmable device being assigned a speed rating number based onits estimated speed. Of course, the estimated speed rating 3462 could bedefined in other ways as well. The technology type 3464 can characterizethe technology of the programmable device in any suitable way. Forexample, FPGAs have been developed based on each of the followingtechnologies: Static Random-Access Memory (SRAM); fuse; antifuse;Programmable Read-Only Memory (PROM); Erasable Programmable Read-OnlyMemory (EPROM); Electrically Erasable Programmable Read-Only Memory(EEPROM); and Flash-erase EPROM technology (Flash). The manufacturer3466 can specify the manufacture of the programmable device. The family3468 can specify the family of the programmable device. Other criteria3470 can include any suitable criteria that can be used to characterizea programmable device, whether currently known or developed in thefuture.

Table 3400 in FIG. 34 includes text labels that describe features thatcould be included in a programmable device. Note, however, the featureset for a programmable device will additionally include a number foreach feature that indicates how many of each feature is available on theprogrammable device. For example, a sample feature set for ahypothetical programmable device could specify 400,000 logic blocks3402; 64 MB memory 3406; and four processors 3410. The feature set for aprogrammable device thus includes all the information about whatresources are available and how many so these resources can be comparedwith the resource requirements of one or more accelerator images.

Referring to FIG. 35 , the accelerator image 480 in FIG. 4 is shown toinclude resource requirements 3500. Resource requirements 3500 specifythe resources the accelerator image 480 needs in order for theaccelerator image to be deployed to a programmable device. Resourcerequirements thus may include, for example, a list of needed resourcesand the quantity of those needed resources. The resource requirements3500 can be compared to the feature set of programmable devices toselect a suitable programmable device so the accelerator image 480 canbe deployed.

Referring to FIG. 36 , a method 3600 is preferably performed by theaccelerator deployment tool 324 shown in FIG. 3 . Resource requirementsfor one or more accelerator images are determined (step 3610). Thefeature sets of available programmable devices are determined (step3620). The resource requirements are then compared to the feature sets(step 3630). When there is no match (step 3640=NO), a message is sentindicating no match (step 3650). When there is a match (step 3640=YES),one or more programmable devices that have feature sets that best matchthe resource requirements for the accelerator image(s) are selected(step 3660). The accelerator image(s) are then deployed to the selectedprogrammable device(s) 3670. Method 3600 is then done. The term “match”as used herein (step 3640=YES) means one or more of the features setssatisfy the resource requirements, and “no match” as used herein (step3640=NO) means none of the feature sets satisfy the resourcerequirements. Note that method 3600 covers the cases of: deploying asingle accelerator image to a single programmable device; deployingmultiple accelerator images to a single programmable device; anddeploying multiple accelerator images to multiple programmable devices.

Determining feature sets of available programmable devices in step 3620in FIG. 36 raises the question of what it means for a programmabledevice to be “available.” Programmable devices may be available based onany suitable availability criteria. Examples of suitable availabilitycriteria are shown in table 3700 in FIG. 37 . Programmable devices onthe same I/O bus as the processor 3702 could be available programmabledevices. Programmable devices on the same server 3710 could be availableprogrammable devices. Programmable devices in the same server group 3720could be available as well. Being in the same server group simply couldmean, for example, the programmable device is available on some serverconnected via a private network to the server running the acceleratordeployment tool. Programmable devices may be available when they are inthe same private cloud 3730, or when they are available in a publiccloud 3740. In addition, combinations of criteria could be used. Forexample, the list shown in FIG. 37 could be ranked such that theprogrammable devices on the same processor I/O bus are considered first.When there are no suitable programmable devices on the same processorI/O bus 3702, programmable devices on the same server 3710 areconsidered. When there are no suitable programmable devices on the sameserver 3710, the programmable devices on other servers in the sameserver group 3720 could be considered. When there are no suitableprogrammable devices on the same server 3710 or in the same server group3720, programmable devices in the same private cloud 3730 could beconsidered. When there are no suitable programmable devices on the sameserver 3710, in the same server group 3720, or in the same private cloud3730, programmable devices in the public cloud 3740 could be considered.The criteria shown in FIG. 37 and the example given above are given byway of example, and are not limiting. A determination of whatprogrammable devices are “available” in step 3620 in FIG. 36 may be madein any suitable way within the scope of the disclosure and claimsherein.

A determination of whether one or more programmable devices matchesresource requirements in step 3640, and the determination of whichprogrammable device(s) best match the resource requirements in step3660, may use suitable match criteria. Examples of suitable matchcriteria are shown in table 3800 in FIG. 38 , and include: first match3810; highest resource usage 3820; lowest usage of constrained resources3830; lowest estimated power consumption 3840; fastest estimated speed3850; lowest monetary cost in a public cloud 3860; estimated reliability3870; and preferred provider 3880. First match 3810 could indicate amatch for the first programmable device considered that satisfies theresource requirements. Highest resource usage 3820 could indicate amatch when a programmable device has the greatest number of itsresources used for a particular accelerator or group of accelerators.For example, to avoid fragmentation of resources, it may be preferablefor one or more accelerators to be deployed to a single programmabledevice that has sufficient resources for the accelerator(s) but thatuses a high percentage of those resources. Multiple programmable devicescould be provided that have varying feature sets, and choosing aprogrammable device that has the highest resource usage for theaccelerator(s) that need to be deployed assures that other programmabledevices that have significantly more resources are not used for theseaccelerator(s). Highest resource usage 3820 could choose a programmabledevice that has enough resources for the accelerator(s) to be deployed,with a minimum of unused resources.

Lowest usage of constrained resources 3830 could indicate a match for aprogrammable device when the accelerator(s) to be deployed have thelowest usage of constrained resources. Constrained resources can bedefined in any suitable way. For example, constrained resources could behard blocks in the programmable device. Constrained resources could alsobe programmable blocks that perform higher-level functions, such asprocessors, DSP slices, network interfaces, multipliers, high-speed I/Ologic, transceivers, Ethernet MAC, bus controllers, external memorycontrollers, disk controllers, etc. By selecting a programmable devicethat has the lowest usage of constrained resources 3830, this means ahigher number of constrained resources in that programmable device willremain available for use by other accelerators.

Lowest estimated power consumption 3840 could indicate a match for aprogrammable device that will have the lowest estimated powerconsumption when programmed with the accelerator(s) that need to bedeployed. Note the estimated power consumption can be a very roughestimate, based, for example, on the programmable device itself withoutconsidering the specifics of the accelerator(s) to be deployed to theprogrammable device. On the other hand, the estimated power consumptioncould be a detailed estimate that takes into account the specifics ofthe accelerator(s) to be deployed, including number and type ofresources used, clock speeds, etc. Of course, estimated powerconsumption could be derived in any suitable way.

Fastest estimated speed 3850 could indicate a match for a programmabledevice that will have the fastest estimated speed when programmed withthe accelerator(s) that need to be deployed. Note the estimated speedcan be a very rough estimate, based, for example, on the programmabledevice itself without considering the specifics of the accelerator(s) tobe deployed to the programmable device. On the other hand, the estimatedspeed could be a detailed estimate that takes into account the specificsof the accelerator(s) to be deployed, including number and type ofresources used, clock speeds, etc. Of course, estimated speed could bederived in any suitable way.

Lowest monetary cost in a public cloud 3860 could indicate a match for aprogrammable device available in a public cloud when the device has thelowest monetary cost when compared to other programmable devices.Estimated reliability 3870 can be based on reliability ratings ofaccelerators by programmers who have used the accelerators. Thereliability ratings allow a programmable device with higher reliabilityratings to be selected over programmable devices with lower reliabilityratings. Preferred provider 3880 allows specifying a provider of theprogrammable device. The provider could be the manufacturer, such asXilinx or Altera. In the alternative, the provider could be a person orcompany that provides programmable devices for deploying accelerators,with different providers having different ratings depending on thenumber of accelerators they have provided, their reliability, userratings, etc. Thus, specifying a preferred provider could cause theselection of programmable devices hosted by a particular preferredprovider over other providers of programmable devices.

FIG. 39 shows a sample system configuration for illustrating some of theprinciples discussed above. We assume a private cloud Cl has two servergroups, Server Group 1 that includes Server 1A and Server 1B, and ServerGroup 2 that includes Server 2A. Server 1A includes two programmabledevices, programmable device 1A-1 that has a corresponding feature set1A-1, and programmable device 1A-2 that has a corresponding feature set1A-2. Server 1B includes a programmable device 1B-1 that has acorresponding feature set 1B-1. Server 2A includes two programmabledevices, programmable device 2A-1 that has a corresponding feature set2A-1, and programmable device 2A-2 that has a corresponding feature set2A-2. The private cloud C1 is connected to a public cloud P1 that hastwo programmable devices, programmable device P1-1 that has acorresponding feature set P1-1, and programmable device P1-2 that has acorresponding feature set P1-2. We assume the features sets include thefeatures shown in FIG. 40 . Note these feature sets in FIG. 40 areextremely simplified for the sake of illustration, and are not intendedto reflect the feature sets of any known programmable devices. We nowconsider several examples to illustrate the selection of a programmabledevice for deploying one or more accelerators.

We assume for a first example a single accelerator image withcorresponding resource requirements, as shown in FIG. 35 . We assume forthis example the resource requirements include: 400,000 logic blocks; 6MB memory; 200 I/O blocks; and one processor. We further assume for thisfirst example and for the configuration in FIG. 39 that an acceleratordeployment tool running on server 1A needs to deploy the acceleratorimage with these resource requirements, and programmable devices are“available” according to an order of priority shown in FIG. 37 . Thus,programmable devices on the same processor I/O bus 3702 will beconsidered first; then programmable devices on the same server 3710 willbe considered; then programmable devices in the same server group 3720will be considered; then programmable devices in the same private cloud3730 will be considered; then programmable devices in a public cloud3740 will be considered. For the sake of simplicity in this firstexample, we also assume a match criteria of selecting the firstprogrammable device that has a feature set that satisfies the resourcerequirements, which corresponds to first match 3810 in FIG. 38 . We alsoassume there are no programmable devices on the same processor I/O bus.We now compare the resource requirements of 400,000 logic blocks, 6 MBmemory, 200 I/O blocks, and one processor against the feature sets inFIG. 40 , starting first with the feature sets 1A-1 and 1A-2, which areon the same server 1A as the accelerator deployment tool that needs todeploy the accelerator. We see that feature set 1A-1 does not have anyprocessors, so programmable device 1A-1 cannot be selected. Feature set1A-2 only has 350,000 logic blocks when the resource requirementsspecify 400,000 logic blocks, so programmable device 1A-2 cannot beselected. So neither of the programmable devices on the same server 1Amatch. Next, we consider servers in the same server group, which isserver 1B. We determine the feature set 1B-1 shown in FIG. 40 providesall of the needed resources in the resource requirements, soprogrammable device 1B-1 is a match. The accelerator deployment toolwill then deploy the accelerator to the programmable device 1B-1.

We assume for a second example there are two accelerator images withcorresponding resource requirements, as shown in FIG. 35 . The firstaccelerator image has resource requirements that include: 200,000 logicblocks, 4 MB memory, 100 I/O blocks, and the second accelerator hasresource requirements that include: 100,000 logic blocks, 2 MB memory,200 I/O blocks, and 1 network connection. We assume for this examplethese two accelerators need to be deployed to the same programmabledevice. By summing these two sets of resource requirements for the twoaccelerators, we arrive at a sum for the two accelerator images of:300,000 logic blocks; 6 MB memory; 300 I/O blocks; and 1 networkconnection. We further assume for this second example and for theconfiguration in FIG. 39 that an accelerator deployment tool running onserver 1A needs to deploy the two accelerator images, and programmabledevices are “available” according to the order of priority shown in FIG.37 . We further assume no programmable devices are on the same processorI/O bus. We also assume a match criteria of first match 3810 in FIG. 38. We now compare the resource requirements of 300,000 logic blocks; 6 MBmemory; 300 I/O blocks; and 1 network connection against the featuresets in FIG. 40 , starting first with the feature sets 1A-1 and 1A-2,which are on the same server 1A as the accelerator deployment tool thatneeds to deploy the accelerator. We see that feature set 1A-1 hassufficient resources to satisfy the summed resource requirements for thetwo accelerators, so programmable device 1A-1 is a match. Theaccelerator deployment tool will then deploy the two accelerators to theprogrammable device 1A-1.

We assume for a third example there is a single accelerator image withcorresponding resource requirements, as shown in FIG. 35 . We assume forthis example the resource requirements include: 500,000 logic blocks; 8MB memory; 400 I/O blocks; one processor; and two network connections.We further assume for this third example and for the configuration inFIG. 39 that an accelerator deployment tool running on server 1A needsto deploy the accelerator image, and programmable devices are“available” according to the order of priority shown in FIG. 37 . Wefurther assume no programmable devices are on the same processor I/Obus. We also assume a match criteria of first match 3810 in FIG. 38 . Wenow compare the resource requirements of 500,000 logic blocks; 8 MBmemory; 400 I/O blocks; one processor; and two network connectionsagainst the feature sets in FIG. 40 , starting first with the featuresets 1A-1 and 1A-2, which are on the same server 1A as the acceleratordeployment tool that needs to deploy the accelerator. We see thatfeature set 1A-1 does not have the needed processor, so programmabledevice 1A-1 does not match. Feature set 1A-2 does not have the needednetwork connections, so programmable device 1A-2 is not a match. Next,we consider programmable device 1B-1 because it is in the same servergroup. We see that feature set 1B-1 in FIG. 40 does not have the needednetwork connections, so programmable device 1B-1 is not a match. We nowconsider programmable devices 2A-1 and 2A-2 because these are in thesame private cloud. Feature set 2A-1 in FIG. 40 does not have the needednetwork connections, so programmable device 2A-1 is not a match. Featureset 2A-2 in FIG. 40 does not have the needed processor, so programmabledevice 2A-2 is not a match. Next, we consider the two programmabledevices P1-1 and P1-2 in the public cloud. Feature set P1-1 lacks theneeded network connections, so programmable device P1-1 is not a match.Feature set P1-2 has all of the needed resources, so the programmabledevice P1-2 is a match. The accelerator deployment tool will then deploythe accelerator to the programmable device P1-2.

We assume for a fourth example there is a single accelerator image withcorresponding resource requirements, as shown in FIG. 35 . We assume forthis example the resource requirements include: 700,000 logic blocks; 32MB memory; 1,000 I/O blocks; two processors; two network connections;and a floating-point math processor. None of the feature sets in FIG. 40satisfy these resource requirements, so there is no match for any of theprogrammable devices corresponding to these feature sets. As a result, amessage will be sent that there is no match, as shown in step 3650 inFIG. 36 .

In the first example above, a first accelerator is deployed toprogrammable device 1B-1, as shown in FIG. 42 . In the second exampleabove, second and third accelerators are deployed to programmable device1A-1, as shown in FIG. 42 . In the third example above, a fourthaccelerator is deployed to programmable device P1-2, as shown in FIG. 42.

A system such as shown in FIG. 39 can be a dynamic system whereaccelerators are added over time. This gives rise to the possibilitythat an accelerator currently deployed might be better matched to a newprogrammable device that has just become available. Referring to FIG. 41, a method 4100 detects availability of a new programmable device (step4110), such as programmable device 1B-2 in FIG. 42 . When a firstaccelerator is not a better match to the new programmable device (step4120=NO), method 4100 is done. When a first accelerator is a bettermatch to the new programmable device (step 4120=YES), an image for thefirst accelerator for the new programmable device is automaticallygenerated (step 4130). The image is then deployed on the newprogrammable device to generate a second accelerator (step 4140). Atthis point in time we have a first accelerator in a first programmabledevice, and a second accelerator in the new programmable device.References that reference the first accelerator are changed to referencethe second accelerator instead (step 4150). The first accelerator isthen cast out of the first programmable device (step 4160). Method 4100is then done. Method 4100 illustrates how an accelerator can bedynamically migrated to a new programmable device when the newprogrammable device becomes available.

We now consider a specific example to illustrate the general principlesdiscussed above with reference to method 4100 in FIG. 41 . FIG. 42 showsthe system in FIG. 39 after the first accelerator was deployed toprogrammable device 1B-1 in Server 1B, after the second and thirdaccelerators were deployed to programmable device 1A-1 in server 1A,after the fourth accelerator was deployed to programmable device P1-2 inpublic cloud P1, and after a new programmable device 1B-2 has been addedto Server 1B. We assume for this example the new programmable device1B-2 has a feature set 1B-2 as shown in FIG. 44 , which includes1,000,000 logic blocks; 32 MB memory; 1,000 I/O blocks; 4 processors;and 6 network connections. We further assume a new programmable deviceis a better match to an existing accelerator when the new programmabledevice is higher in a defined hierarchy of availability criteria thanthe first programmable device. Thus, assuming the availability criteria3700 shown in FIG. 37 , a new programmable device is a better match toan existing accelerator when the new programmable device has a featureset that satisfies the resource requirements of the existing acceleratorand when the new programmable device is at a higher level in thehierarchy shown in FIG. 37 . Thus, a new programmable device on the sameserver 3710 that has a feature set that satisfies the resourcerequirements of an accelerator is a better match than the programmabledevice where an accelerator is currently deployed in a same server group3720. The availability criteria 3700 can thus be used in determiningwhen a new programmable device is a better match then the existingprogrammable device for an accelerator. Availability criteria 3700,however, is not the only criteria that can be considered. Other factors,including match criteria 3800 shown in FIG. 38 , can be used todetermine when a new accelerator is a better match for an existingaccelerator than the existing accelerator's programmable device.

We assume the accelerator deployment tool detects the addition of thenew programmable device 1B-2 shown outlined in bold in FIG. 42 (step4110 in FIG. 41 ). Next, method 4100 determines whether an existingaccelerator is a better match to the new programmable device than to theaccelerator's current programmable device (step 4120). For the examplein FIG. 42 and discussed above as the first example, we assumed thefirst accelerator has resource requirements that include: 400,000 logicblocks; 6 MB memory; 200 I/O blocks; and one processor. We compare theseresource requirements to the features set shown in FIG. 44 , anddetermine the feature set of the new programmable device satisfies theresource requirements for the first accelerator. We assume for thissimple example the hierarchy of availability criteria 3700 shown in FIG.37 is the primary measure of whether a new programmable device is abetter match for an existing accelerator than the programmable device onwhich the existing accelerator is currently deployed. The firstaccelerator that was deployed to programmable device 1B-1 is on the sameserver as the new programmable device 1B-2, so the new programmabledevice is not a better match for the first accelerator because the newprogrammable device is not higher in the hierarchy of availabilitycriteria shown in FIG. 37 . The second and third accelerators in thesecond example above that were deployed to programmable device 1A-1 havea sum of resource requirements that include: 300,000 logic blocks; 6 MBmemory; 300 I/O blocks; and 1 network connection. While the newprogrammable device 1B-2 has a feature set 1B-2 that satisfies theresource requirements for these two accelerators, the second and thirdaccelerators deployed to programmable device 1A-1 are on the same serveras the accelerator deployment tool. Same server 3710 in FIG. 37 ishigher in the hierarchy of availability criteria than same server group3720, so the new programmable device is not a better match for thesecond and third accelerators. The fourth accelerator deployed in thethird example above to the programmable device P1-2 in public cloud P1has resource requirements that include: 500,000 logic blocks; 8 MBmemory; 400 I/O blocks; one processor; and two network connections. Thenew programmable device 1B-2 has a feature set 1B-2 that satisfies theseresource requirements for the fourth accelerator. Because the newprogrammable device 1B-2 is in the same server group as server 1A, whichhosts the accelerator deployment tool, the new programmable device 1B-2is a better match for the fourth accelerator (step 4120=YES). Inresponse, an accelerator image for the fourth accelerator isautomatically generated (step 4130), the accelerator image is deployedto the new programmable device to generate a new fifth accelerator (step4140), and references in the code to the fourth accelerator are changedto reference the fifth accelerator instead (step 4150). For example, inthe virtual function table 2600 shown in FIG. 33 , the reference to thefourth accelerator (Acc4) is replaced by a reference to the fifthaccelerator (Acc5). The fourth accelerator is then cast out of theprogrammable device P1-2 (step 4160). The result is the configuration inFIG. 43 , where the fifth accelerator in the new programmable device1B-2 has replaced the fourth accelerator that was deployed toprogrammable device P1-2, and which has been cast out of theprogrammable device P1-2.

While the specific example discussed above used the ranked hierarchy ofavailability criteria in FIG. 37 as one suitable criteria fordetermining whether a new programmable device is better suited to anexisting accelerator, any suitable criteria could be used to make thisdetermination, including the match criteria shown in FIG. 38 , or anyother suitable criteria, all of which are within the scope of thedisclosure and claims herein.

The addition of a new programmable device may make it possible to deploymultiple existing accelerators to the new programmable device. Forexample, the feature set 1B-2 for the new programmable device 1B-2 shownin FIG. 42 satisfies the sum of the resource requirements for both thefirst accelerator in programmable device 1B-1 and the fourth acceleratorin programmable device P1-2. We assume some match criteria is definedthat indicates the new programmable device 1B-2 is a better match forboth the first accelerator and the fourth accelerator. FIG. 45 shows onesuitable method 4500 that can be performed by the accelerator deploymenttool when multiple existing accelerators could be deployed to a newprogrammable device (step 4510). The accelerators to deploy to the newprogrammable device are selected (step 4520). In the example in FIG. 42, we assume the first accelerator in programmable device 1B-1 and thefourth accelerator in programmable device P1-2 are selected fordeployment to the new programmable device 1B-2. Accelerator images forthe first accelerator and the fourth accelerator on the new programmabledevice 1B-2 are automatically generated (step 4530). Thesenewly-generated images are then deployed to the new programmable deviceto generate the new accelerators (step 4540). This means in FIG. 42 afifth accelerator and sixth accelerator are deployed to the newprogrammable device 1B-2 as replacements for the first accelerator andfourth accelerator, respectively. The references are changed toreference the new accelerators instead of the old accelerators (step4550). This means references to the first accelerator will be replacedwith references to the fifth accelerator, and references to the fourthaccelerator will be replaced with references to the sixth accelerator.This could happen, for example, by replacing references to the first andfourth accelerators in a virtual function table similar to that shown inFIG. 33 with references to the fifth and sixth accelerators,respectively. At this point, there will be no references to the first orfourth accelerators, because these have all been replaced by referencesto the fifth and sixth accelerators, respectively. The old acceleratorsare cast out of the programmable devices (step 4560). This means thefirst accelerator is cast out of the programmable device 1B-1 and thefourth accelerator is cast out of the programmable device P1-2. Theresult is a fifth and sixth accelerator in the new programmable device1B-2 have replaced the first accelerator in programmable device 1B-1 andthe fourth accelerator in the programmable device P1-2. Method 4500 isthen done.

The examples above are extremely simplified for the purpose ofillustrating the general concepts herein. What constitutes availableprogrammable devices and what constitutes a match can be defined anysuitable way, including those specific ways shown in FIGS. 37 and 38 aswell as another other suitable way. The disclosure and claims hereinextend to any suitable way to match programmable devices to resourcerequirements based on the feature sets for the programmable devices, andto any suitable criteria for determining when a new programmable deviceis a better match to one or more existing accelerators in otherprogrammable devices.

The accelerators shown in FIGS. 8, 15, 18, 22, 29 and 32 include anOpenCAPI interface. Note, however, the OpenCAPI interface is notstrictly necessary to dynamically generate and deploy an accelerator asdisclosed and claimed herein. Deploying an accelerator to a programmabledevice that includes an OpenCAPI interface is useful because theOpenCAPI specification is open, allowing anyone to develop to thespecification and interoperate in a cloud environment. In addition, theOpenCAPI interface provides lower latency, reducing the “distance”between an accelerator and the data it may consume or produce.Furthermore, OpenCAPI provides higher bandwidth, increasing the amountof data an accelerator can consume or produce in a given time. Theseadvantages of OpenCAPI combine to provide a good environment forimplementing a code portion of a computer program in an accelerator, andto lower the threshold for a code portion to be better in an acceleratorthan in the computer program. However, the disclosure and claims hereinapply equally to accelerators that do not include or have access to anOpenCAPI interface.

An accelerator deployment tool deploys multiple accelerators to multipleprogrammable devices, and detects when a new programmable device becomesavailable. When a first accelerator in a first programmable device is abetter match to the new programmable device, the accelerator deploymenttool automatically generates an image for the first accelerator for thenew programmable device, deploys the image on the new programmabledevice to generate a second accelerator, changes references to the firstaccelerator to reference instead the second accelerator, and casts thefirst accelerator out of the first programmable device.

One skilled in the art will appreciate that many variations are possiblewithin the scope of the claims. Thus, while the disclosure isparticularly shown and described above, it will be understood by thoseskilled in the art that these and other changes in form and details maybe made therein without departing from the spirit and scope of theclaims.

The invention claimed is:
 1. An apparatus comprising: at least oneprocessor; a memory coupled to the at least one processor; a pluralityof accelerators deployed to a plurality of programmable devices; and anaccelerator deployment tool residing in the memory and executed by theat least one processor, the accelerator deployment tool performing thefollowing steps: detecting availability of a new programmable device,while a first accelerator of the plurality of accelerators is executingon a first programmable device, determining that the first acceleratoris a better match to the new programmable device than to the firstprogrammable device, and in response, automatically generating an imagefor the first accelerator for the new programmable device, deploying theimage on the new programmable device to generate a second accelerator,changing references to the first accelerator to reference instead thesecond accelerator, and casting the first accelerator out of the firstprogrammable device, wherein the accelerator deployment tool determinesthe first accelerator is a better match to the new programmable devicewhen the new programmable device is higher in a defined hierarchy ofavailability criteria than the first programmable device and bycomparing a feature set for the new programmable device with resourcerequirements for the first accelerator, the feature set comprising atleast one analog feature that is programmable.
 2. The apparatus of claim1 wherein the plurality of programmable devices each comprises an OpenCoherent Accelerator Processor Interface (OpenCAPI).
 3. The apparatus ofclaim 1 wherein the feature set for the new programmable devicecomprises a plurality of digital features that are programmable and atleast one fixed feature.
 4. The apparatus of claim 3 wherein theplurality of digital features comprises at least two of the following:logic blocks; flip-flops; memory; input/output (1/0) blocks; processors;and network interfaces.
 5. The apparatus of claim 4 wherein theplurality of digital features further comprises at least two of thefollowing: digital signal processing (DSP) slices; multipliers;high-speed 1/0 logic; transceivers; Ethernet media access controls(MACs); bus controllers; and external memory controllers.
 6. Theapparatus of claim 1 wherein the first accelerator is a better match tothe new programmable device when the feature set for the newprogrammable device better satisfies the resource requirements for thefirst accelerator based on at least one of the following match criteria:first match; highest usage of resources; lowest usage of constrainedresources; lowest estimated power consumption; fastest estimated speed;lowest monetary cost in a public cloud; estimated reliability; andpreferred provider.
 7. An apparatus comprising: at least one processor;a memory coupled to the at least one processor; a plurality ofaccelerators deployed to a plurality of programmable devices that eachhas a corresponding feature set that comprises: an Open CoherentAccelerator Processor Interface (OpenCAPI); a plurality of digitalfeatures that are programmable comprising: logic blocks; memory;input/output (I/O) blocks; processors; and network interfaces; at leastone fixed feature; and at least one analog feature that is programmable;an accelerator deployment tool residing in the memory and executed bythe at least one processor, the accelerator deployment tool performingthe following steps: detecting availability of a new programmabledevice, while a first accelerator of the plurality of accelerators isexecuting on a first programmable device, determining that the firstaccelerator is a better match to the new programmable device than to thefirst programmable device, and in response, automatically generating animage for the first accelerator for the new programmable device,deploying the image on the new programmable device to generate a secondaccelerator, changing references to the first accelerator to referenceinstead the second accelerator, and casting the first accelerator out ofthe first programmable device, wherein the accelerator deployment tooldetermines the first accelerator is a better match to the newprogrammable device by comparing a feature set for the new programmabledevice with resource requirements for the first accelerator, the featureset comprising at least one analog feature that is programmable, whenthe new programmable device is higher in a defined hierarchy ofavailability criteria than the first programmable device, and based onat least one of the following match criteria: first match; highest usageof resources; lowest usage of constrained resources; lowest estimatedpower consumption; fastest estimated speed; lowest monetary cost in apublic cloud; estimated reliability; and preferred provider.
 8. Theapparatus of claim 7 wherein the plurality of digital features furthercomprises at least one of the following: digital signal processing (DSP)slices; multipliers; high-speed I/O logic; transceivers; Ethernet mediaaccess controls (MACs); bus controllers; and external memorycontrollers.
 9. A method for deploying an accelerator, the methodcomprising: providing a plurality of accelerators deployed to aplurality of programmable devices; detecting availability of a newprogrammable device; while a first accelerator of the plurality ofaccelerators is executing on a first programmable device, determiningthat the first accelerator is a better match to the new programmabledevice than to the first programmable device by determining when the newprogrammable device is higher in a defined hierarchy of availabilitycriteria than the first programmable device, and in response:automatically generating an image for the first accelerator for the newprogrammable device; deploying the image on the new programmable deviceto generate a second accelerator; changing references to the firstaccelerator to reference instead the second accelerator; and casting thefirst accelerator out of the first programmable device whereindetermining that the first accelerator is a better match to the newprogrammable device further comprises comparing a feature set for thenew programmable device with resource requirements for the firstaccelerator, the feature set comprising at least one analog feature thatis programmable.
 10. The method of claim 9 wherein the plurality ofprogrammable devices each comprises an Open Coherent AcceleratorProcessor Interface (OpenCAPI).
 11. The method of claim 9 wherein thefeature set for the new programmable device comprises a plurality ofdigital features that are programmable and at least one fixed feature.12. The method of claim 11 wherein the plurality of digital featurescomprises at least two of the following: logic blocks; flip-flops;memory; input/output (11O) blocks; processors; and network interfaces.13. The method of claim 12 wherein the plurality of digital featuresfurther comprises at least two of the following: digital signalprocessing (DSP) slices; multipliers; high-speed 11O logic;transceivers; Ethernet media access controls (MACs); bus controllers;and external memory controllers.
 14. The method of claim 9 wherein thefirst accelerator is a better match to the new programmable device whenthe feature set for the new programmable device better satisfies theresource requirements for the first accelerator based on at least one ofthe following match criteria: first match; highest usage of resources;lowest usage of constrained resources; lowest estimated powerconsumption; fastest estimated speed; lowest monetary cost in a publiccloud; estimated reliability; and preferred provider.